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Before corresponding points in images taken with two cameras can be 
iver distances to objects in a scene, one has to determine the position and 
of one camera relative to the other. This is the classic photogrammetric 
relative orientation, central to the interpretation of binocular stereo 
Iterative methods for determining relative orientation were developed 
without them we would not have most of the topographic maps we do 
ative orientation is also of importance in the recovery of motion and 
an image sequence when successive frames are widely separated in time. 
Jmotion vision are rediscovering some of the methods of photogrammetry. 
ed here is a particularly simple iterative scheme for recovering relative 
that, unlike existing methods, does not require a good initial guess for 
and the rotation. The data required is a set of pairs of corresponding 
e two projection centers to points in the scene. It is well known that at 
.irs of rays are needed. Less appears to be known about the existence of 
|utions and their interpretation. These issues are discussed here in detail, 
iguous determination of all of the parameters of relative orientation is 
when the observed points lie on a critical surface. These surfaces and 
lerate forms are analysed here as well. 
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1. Introduction 

The positions of corresponding points in two images can be used to determine 
the positions of points in the environment, provided that the position and 
orientation of one camera with respect to the other is known. Given the 
internal geometry of the camera, including its focal length and the location of 
the principal point, rays can be constructed by connecting the points in the 
images to their corresponding projection centers. These rays, when extended, 
intersect at the point in the scene that gave rise to the image points. This is 
how binocular stereo data is used to determine the positions of points in the 
environment after the correspondence problem has been solved. 

It is also the method used in motion vision when feature points are tracked 
and the image displacements that occur in the time between two successive 
frames are relatively large (see for example [Ullman 1979] and [Tsai & Huang 
1984]). The connection between these two problems has not attracted much 
attention before, nor has the relationship of motion vision to some aspects 
of photogrammetry (but see [Longuet-Higgins 1981]). It turns out, for exam¬ 
ple, that the well known motion field equations [Longuet-Higgins & Prazdny 
1980, Bruss & Horn 1983] are just the parallax equations of photogrammetry 
[Hallert 1960, Moffit & Mikhail 1980] that occur in the incremental adjust¬ 
ment of relative orientation. Most papers on relative orientation only give the 
equation for y-parallax, corresponding to the equation for the y-component of 
the motion field (see for example the first equation in [Gill 1964], equation (1) 
in [Jochmann 1965], and equation (6) in [Oswal 1967]). Some papers actually 
give equations for both x- and y-parallax (see for example equation (9) in 
[Bender 1967]). 

In both binocular stereo and large displacement motion vision analysis, 
it is necessary to first determine the relative orientation of one camera with 
respect to the other. The relative orientation can be found if a sufficiently large 
set of pairs of corresponding image points have been identified [Thompson 
1959b, Thompson 1968, Ghosh 1972, Schwidefsky 1973, Slama et al. 1980, 
Moffit & Mikhail 1980, Wolf 1983, Horn 1986]. 

Let us use the terms left and right to distinguish the two cameras (in the 
case of the application to motion vision these will be the camera positions and 
orientations corresponding to the earlier and the later frames respectively). 1 
The ray from the center of projection of the left camera to the center of 
projection of the right camera is called the baseline (see Fig. 1). A coordinate 
system can be erected at each projection center, with one axis along the 

x In what follows we use the coordinate system of the right (or later) camera as 
the reference. One can simply interchange left and right if it happens to be more 
convenient to use the coordinate system of the left (or earlier) camera. 
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optical axis, that is, perpendicular to the image plane. The other two axes 
are in each case parallel to two convenient orthogonal directions in the image 
plane (such as the edges of the image, or lines connecting pairs of fiducial 
marks). The rotation of the left camera coordinate system with respect to 
the right is called the orientation. 
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Figure 1. Points in the environment are viewed from two camera positions. 
The relative orientation is the direction of the baseline b, and the rotation 
R relating the left and right coordinate systems. The directions of rays 
to at least five scene points must be known in both camera coordinate 
systems. 


Note that we cannot determine the length of the baseline without knowledge 
about the length of a line in the scene, since the ray directions are unchanged 
if we scale all of the distances in the scene and the baseline by the same 
positive scale-factor. This means that we should treat the baseline as a unit 
vector, and that there are really only five unknowns—three for the rotation 
and two for the direction of the baseline. 2 


2. Existing Solution Methods 

Various empirical procedures have been devised for determining the relative 
orientation in an analog fashion. Most commonly used are stereoplotters, op- 

2 If we treat the baseline as a unit vector, its actual length becomes the unit of 
length for all other quantities. 
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tical devices that permit viewing of image pairs and superimposed synthetic 
features called floating marks. Differences in ray direction parallel to the 
baseline are called horizontal disparities (or x-parallaxes), while differences in 
ray direction orthogonal to the baseline are called vertical disparities (or y- 
parallaxes). 3 Horizontal disparities encode distances to points on the surface 
and are the quantities sought after in measurement of the underlying topogra¬ 
phy. There should be no vertical disparities when the device is adjusted to the 
correct relative orientation, since the rays from the left and right projection 
center must lie in a plane that contains the baseline if they are to intersect. 

The methods used in practice to determine the correct relative orien¬ 
tation depend on successive adjustments to eliminate the vertical disparity 
at each of five or six carefully chosen points [Sailor 1960, Thompson 1964, 
Slama et al. 1980, Moffit & Mikhail 1980, Wolf 1974]. In each of the ad¬ 
justments a single parameter of the relative orientation is varied in order to 
remove the vertical disparity at one of the points. Which adjustment is made 
to eliminate the vertical disparity at a particular point depends on the partic¬ 
ular method is chosen. In each case, however, one of the adjustments, rather 
than being guided visually, is made by an amount that is calculated, using 
the measured values of earlier adjustments. The calculation is based on the 
assumptions that the surface being viewed can be approximated by a plane, 
that the baseline is roughly parallel to this plane, and that the optical axes of 
the two cameras are roughly perpendicular to this plane. The whole process 
is iterative in nature, since the reduction of vertical disparity at one point 
by means of an adjustment of a single parameter of the relative orientation 
disturbs the vertical disparity at the other points. Convergence is usually 
rapid if a good initial guess is available. It can be slow, however, when the 
assumptions on which the calculation is based are violated, such as in “acci- 
dented” or hilly terrain [Van Der Weele 1959-60]. These methods typically 
use Euler angles to represent three-dimensional rotations [Korn & Korn 1968] 
(traditionally denoted by the greek letters k, <f>, and a;). Euler angles have 
a number of short-comings for describing rotations that become particularly 
noticable when these angles become large. 

There also exist related digital procedures that converge rapidly when a 
good initial guess of the relative orientation is available, as is usually the case 
when one is interpreting aerial photography [Slama et al. 1980]. Not all of 
these methods use Euler angles. Thompson [1959b], for example, uses twice 
the Gibb’s vector [Korn & Korn 1968] to represent rotations. These proce- 

3 This naming convention stems from the observation that, roughly speaking, in 
the usual viewing arrangement, horizontal disparities correspond to left-right 
displacements in the image, whereas vertical disparities correspond to up-down 
displacements. 
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dures may fail to converge to the correct solution when the initial guess is far 
off the mark. In the application to motion vision, approximate translational 
and rotational components of the motion are often not known initially, so a 
procedure that depends on good initial guesses is not particularly useful. 

Normally, the directions of the rays are obtained from points in images 
generated by projection onto a planar surface. In this case the directions are 
confined to the field of view as determined by the active area of the image plane 
and the distance to the center of projection. The field of view is always less 
than a hemi-sphere, since only points in front of the camera can be imaged. 4 
The method described here applies, however, no matter how the directions 
to points in the scene are determined. There is no restriction on the possible 
directions of the rays. We do assume, however, that we can tell which of two 
opposite semi-infinite line-segments the point lies on. If a point lies on the 
correct line-segment we will say that it lies in front of the camera, otherwise 
it will be considered to be behind the camera. 

The problem of relative orientation is generally considered solved, and 
so has received little attention in the photogrammetric literature in recent 
times [Van Der Weele 1959-60]. In the annual index of Photogrammetric 
Engineering , for example, there is only one reference to the subject in the last 
ten years [Ghilani 1983] and six in the decade before that. This is very little 
in comparison to the large number of papers on this subject in the fifties, 
as well as the sixties, including [Gill 1964], [Sailor 1965], [Jochmann 1965], 
[Ghosh 1966], [Forrest 1966] and [Oswal 1967]. 

In this paper we discuss the relationship of relative orientation to the 
problem of motion vision in the situation where the motion between the ex¬ 
posure of successive frames is relatively large. Also, a new iterative algorithm 
is described here, as well as a way of dealing with the situation when there 
is no initial guess available for the rotation or the direction of the baseline. 
The advantages of the unit quaternion notation for representing rotations are 
illustrated as well. Finally, we discuss critical surfaces, surface shapes that 
lead to difficulties in establishing a unique relative orientation. 

3. Coplanarity Condition 

If the ray from the left camera and the corresponding ray from the right 
camera are to intersect, they must to lie in a plane that also contains the 
baseline. Thus, if b is the vector respresenting the baseline, r r the ray from 
the right projection center to the point in the scene and r/ the ray from the 

4 The field of view is larger than a hemi-sphere in some fish-eye lenses, where there 
is significant radial distortion. 
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left projection center to the point in the scene, then the triple product 

[b r} r r ] 

equals zero, where rj = Rot(rj) is the left ray rotated into the right camera’s 
coordinate system. 5 This is the coplanarity condition (see Fig. 2). 


N / 

\ / 



Figure 2. Two rays approach closest where they are intersected by a 
line perpendicular to both. If there is no measurement error, and the 
relative orientation has been recovered correctly, then the two rays actually 
intersect. In this case the two rays and the baseline lie in a common plane. 


We obtain one such constraint from each pair of rays. There will be an infinite 
number of solutions for the baseline and the rotation when there are fewer 
than five pairs of rays, since there are five unknowns and each pair of rays 
yields only one constraint. Conversely, if there are more than five pairs of 
rays, the constraints are likely to be inconsistent as the result of small errors 
in the measurements. In this case, no exact solution of the set of constraint 
equations exists, and it makes sense instead to minimize the sum of squares of 
errors in the constraint equations. In practice, one should use more than five 
pairs of rays in order to reduce the influence of measurement errors [Jochmann 
1965, Ghosh 1966]. We shall see later that the added information also allows 
one to eliminate spurious solutions. 

5 The baseline vector b is here also measured in the coordinate system of the right 


camera. 
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4. What is the Appropriate Error Term? 

The triple product [b rj r r ] is zero when the left and right ray are coplanar 
with the baseline. It is not immediately apparent, however, that the triple 
product itself is necessarily the ideal measure of departure from best fit. It is 
worthwhile exploring the geometry of the two rays more carefully. Consider 
the points on the rays where they approach each other the closest (see Fig. 2). 
The line connecting these points will be perpendicular to both rays, and hence 
parallel to (rj x r r ). As a consequence, we can write 

<* rj + 7 (rj x r r ) = b + ft r r , 

where a and ft are proportional to the distances along the left and the right ray 
to the points where they approach most closely, while 7 is proportional to the 
shortest distance between the rays. We can find 7 by taking the dot-product 
of the equality above with rj x r r . We obtain 

7 II r j X rr || 2 = [b rj r r ]. 

Similarly, taking the dot-products with r r x (rj x 17 ) and rj x (rj x ry), we 
obtain 

a ||rj x r r || 2 = (b x r r ) • (rj x r r ), 
ft ||rj x r r || 2 = (b x rj) • (rj x r r ). 

The magnitudes of a and ft are the distances along the rays to the point of 
closest approach when r r and rj are unit vectors. It turns out, however, that 
we are more interested here in the signs than in the magnitudes of a and ft. 

Normally, the points where the two rays approach the closest will be in 
front of both cameras, that is, both a and ft will be positive. If the estimated 
baseline or rotation is in error, however, then it is possible for one or both of 
the calculated parameters a and ft to be negative. We will use this observation 
later to distinguish amongst different apparent solutions. 

We have shown that the perpendicular distance between the left and the 
right ray is equal to ratio of the triple product [b rj r r ] to the magnitude 
squared of (rj x r r ). But the measurement errors are in the image, not in 
the scene. Thus a least-squares procedure should be based on an error in 
determining the direction of the rays, not on the distance of closest approach. 
To arrive at such a measure, we can look at the angle, 8 say, between the 
projections of the left and right ray into a plane perpendicular to b (see 
Fig. 3). This angle will be zero when the vertical disparity is zero, that is, 
when the left and right rays are coplanar with the baseline. 

The projections of rj and r r into a plane perpendicular to b are given by 

rj = rj - (rj • b)b = (b x rj) x b, 
r r = r r — (r r • b)b = (b x r r ) x b, 
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Figure 3. One measure of the departure from the coplanarity condition 
is the angle 6 between the planes formed by the left ray and the baseline 
and the plane formed by the right ray and the baseline. The angle may 
be found by projection the two rays onto a plane perpendicular to the 
baseline. 


where we have used the fact that b is a unit vector. It is easy to show that 

11311 = ll b x 311 ll*v|| = ||b x r r ||, 


keeping in mind again that b is a unit vector, and that b x rj and bxr r are 
perpendicular to b. The cross-product of the two projected vectors rj and r r 
will be parallel to the baseline and have magnitude proportional to the sine 
of the angle between the projected vectors, and so, 


sin# = 


[b r} r r 

II3IIIM 


[b r'f r r ] 

b x rj|| ||b x r r 


where we have used the fact that [b r\ ly] = [b rj r r ], something that can 
easily be verified. 
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We could use the sine of the angle between the projected vectors directly 
as a measure of departure from best fit. This is not as good an idea as it may 
appear at first sight, because of what happens when one of the rays becomes 
nearly parallel to the baseline. In this case the angle will vary rapidly with 
small changes in the direction of the ray. Correspondingly, one of the terms 
in the denominator in the expression for the sine of the angle becomes small. 
It is better to normalize the expression by multiplying by the lengths of the 
projected vectors. Then we obtain the area of the parallelogram formed by 
the projections of the rays into a plane perpendicular to the baseline, namely, 

11311 IM sin 6 = [b r} r r ] = [b r{ r r ]. 

This discussion confirms that the triple product itself is a good measure of the 
departure from best fit. This is convenient, since it makes the least squares 
analysis reasonably straightforward. If one desires to use a different error 
measure, one can weight the terms in the sums to follow accordingly. 


5. Least Squares Solution for the Baseline 


If the rotation is known, it is easy to find the best fit baseline, as we show next. 
This is useful, despite the fact that we do not usually know the rotation. The 
reason is that the ability to find the best baseline, given a rotation, reduces 
the dimensionality of the search space from five to three. 

Let {r/^} and {r r) ,}, for i = 1 ...n, be corresponding sets of left and 
right rays. We wish to minimize 


E = J2[b r\ ti r r>i ] 2 = ]T(b • (r' M 


t=l 


i=l 


x r 



subject to the condition ||b|| 2 = 1, where r{ i is the rotated left ray r j t i, as 
before. If we let Cj = r{ i x r r; ;, we can rewrite the sum in the simpler form 


E - J^(b • c t ) 2 = b r (^ acf ) b, 

i=i V i=i / 

where we have used the equivalence b • c,- = b r Cj, which depends on the 
interpretation of column vectors as 3 x 1 matrices. The term C(cJ is a dyadic 
product, a 3 x 3 matrix obtained by multiplying a 3 x 1 matrix by a 1 X 3 
matrix. 

The error sum is a quadratic form involving the real symmetric matrix 

c=i>ej\ 

i=l 

The minimum of such a quadratic form is the smallest eigenvalue of the matrix 
C, attained when b is the corresponding unit eigenvector (see, for example, 
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the discussion of Rayleigh’s quotient in [Korn & Korn 1968]). This can be 
verified by introducing a Lagrangian multiplier A and minimizing 

E' = b T Cb + \(l-b T b), 

subject to the condition b T b = 1. Differentiating with respect to b and 
setting the result equal to zero yields: 

Cb = Ab. 

The error corresponding to a particular solution of this equation is found by 
premultiplying by b T : 

E = b T C b = Ab T b = A. 

The three eigenvalues of the real symmetric matrix C are non-negative, and 
can be found in closed form by solving a cubic equation, while each of the 
corresponding eigenvectors has components that axe the solution of three ho¬ 
mogeneous equations in three unknowns [Korn & Korn 1968]. If the data 
are relatively free of measurement error, then the smallest eigenvalue will 
be much smaller than the other two, and a reasonable approximation to the 
sought-after result can be obtained by solving for the eigenvector using the 
assumption that the smallest eigenvalue is actually zero. This way one need 
not even solve the cubic equation (see also [Horn & Weldon 1988]). 

If b is a unit eigenvector, so is — b. Changing the sense of the baseline 
does not change the magnitude of the error term [b r| r r ]. It does, however, 
change the signs of a , ft and 7 . One can decide which sense of the baseline 
direction is appropriate by determining the signs of ai and fti for i = 1. .. n. 
Ideally, they should all be positive. The solution for the optimal baseline is 
not unique unless there are at least two pairs of corresponding rays. The 
reason is that the eigenvector we are looking for is not uniquely determined 
if more than one of the eigenvalues is zero, and the matrix has rank less than 
two if it is the sum of fewer than two dyadic products of independent vectors. 
This is not a significant restriction, however, since we need at least five pairs 
of rays to solve for the rotation anyway. 

6 . Iterative Improvement of Relative Orientation. 

If one ignores the orthonormality of the rotation matrix, a set of nine homoge¬ 
neous linear equations can be obtained by a transformation of the coplanarity 
conditions that was first described in [Thompson 1959b]. These equations 
can be solved when eight pairs of corresponding ray directions are known 
[Longuet-Higgins 1981]. This is not a least-squares method that can make 
use of redundant measurements, nor can it be applied when fewer than eight 
points are given. The method is also strongly affected by measurement errors 
and fails for certain configurations of points [Longuet-Higgins 1984]. 
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No true closed-form solution of the least-squares problem has been found 
for the general case, where both baseline and rotation are unknown. However, 
it is possible to determine how the overall error is affected by small changes 
in the baseline and small changes in the rotation. This allows one to make 
iterative adjustments to the baseline and the rotation that reduce the sum of 
squares of errors. 

We can represent a small change in the baseline by an infinitesimal quan¬ 
tity 5b. If this change is to leave the length of the baseline unaltered, then 

||b + 5b|| 2 = ||b|| 2 , 

or 

||b|| 2 + 2b • 5b + ||5b|| 2 = ||b|| 2 . 

If we ignore quantities of second-order, we obtain 

5b • b = 0, 

that is, 5b must be perpendicular to b. 

A small change in the rotation can be represented by an infinitesimal 
rotation vector 5a;. The direction of this vector is parallel to the axis of rota¬ 
tion, while its magnitude is the angle of rotation. This incremental rotation 
takes the rotated left ray, rj, into 

r" = r| + (5a; x rj). 

This follows from Rodrigues’ formula for the rotation of a vector r: 

cos 6 r + sin ^(wxr)-t-(l- cos 9){u • r)a;, 
if we let 9 = ||5o;||, a; = 5a;/ ||5o;||, and note that 5 a; is an infinitesimal 
quantity. Finally then, we see that [b rj r r ] becomes 

[(b + 5b) (rj + 5a; x r{) r r ], 


or, 

[b r{ r r ] + [5b r| r r ] + [b (5a; x r{) r r ], 
if we ignore the term [5b (5a; x r[) r r ], because it contains the product of 
two infinitesimal quantities. We can expand two of the triple products in the 
expression above and obtain 

[b r} r r ] -f (r[ x r r ) • 5b + (r r x b) • (5a; x rj), 


or 

for short, where 

< = [br[ r r ], 


t + c • 5b + d • 5a;, 

c = rj x r r , and d = rj x (r r x b). 


Now, we are trying to minimize 

n 

E = ^2 (*» + Cj • 5b -I- dj ■ 5a;) 2 , 
1=1 
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subject to the condition b • <5b = 0. This constrained minimization problem 
can be transformed into an equivalent unconstrained form by introduction 
of a Lagrangian multiplier. Instead of minimizing E itself, we then have to 
minimize: 

E' = E + A(b • <5b). 

Differentiating E' with respect to (5b, and setting the result equal to zero 
yields, 

n 

(ti + c* • 6b + d; • Su) Cj + Ab = 0. 

2=1 

Before we can proceed, we need to eliminate the unknown Lagrangian multi¬ 
plier A. The dot-product of the expression with b leads to 

n 

y (ti + Cj • <5b + d, • Su) Cj • b + A(b • b) = 0, 

2=1 

which, since b • b = 1, gives us a value for A that we can use to compute the 
term 

n 

Ab = —b (ti -f c,- • <Sb + dj • Su) c * • b, 

1=1 

or 

n 

Ab = —(bb r ) ^ (ti + Ci • <5b + d,- • Su) c , . 

2=1 

Finally, substituting for Ab in the equation above we obtain 

n 

B (ti + c,- • 6 b + d; • Siv) c i = 0, 

2=1 

where 

B = (I — bb 7 ’) 

is the projection operator that removes components of vectors parallel to the 
baseline, and I is just the 3x3 identity matrix. We can conclude that the 
equation above relates quantities in a plane perpendicular to the baseline. 

Finally, if we differentiate E' with respect to 8u and set this result also 
equal to zero, we obtain 

n 

(ti + Ci • <5b + d,- • Siv) d; = 0. 

t=i 

Together, the two vector equations constitute six linear scalar equations in 
the six unknown components of <5b and Su. We can rewrite them in the more 
compact form: 

BC 8b + BF Su} = -Be 
F T Sb + D Su = -d 
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or 

(BC BF\ ( 8h\ _ _ (Bc\ 

\f t d){s u J- Ui 

where 

C = ^2acf, F = Y / c i df, and £> = ^d l df, 

t=l t=l i=l 

while 

n n 

« = and d = Y, i id i . 

i=l i=l 

The matrix B is singular, as can be seen by noting that b is an eigenvector with 
zero eigenvalue. Thus the first three equations above are not independent. 
One of them will have to be removed. Fortunately, we also still have to 
incorporate the condition that 6 b be perpendicular to b. We can do this by 
replacing one of the first three equations with the linear equation b • 6b = 0. 
For the best numerical accuracy one should eliminate the equation with the 
smallest coefficients. 

The above gives us a way of finding small changes in the baseline and 
rotation that reduce the overall error sum. This method can be applied itera¬ 
tively to locate a minimum. Numerical experiments confirm that it converges 
rapidly when a good initial guess is available. Incremental adjustments can¬ 
not be computed if the six-by-six coefficient matrix becomes singular. This 
will happen when there are fewer than five pairs of rays, and for certain rare 
configurations of points in the scene (see the discussion of critical surfaces 
later on). 

7 . Adjusting the Baseline and the Rotation 

The iterative adjustment of the baseline is straightforward: 

b n+i = b ” + 6b”, 

where b n is the baseline estimate at the beginning of the n-th iteration, while 
6 b n is the adjustment computed during the n-th iteration, as discussed in 
the previous section. If 6 b” is not infinitesimal, the result will not be a unit 
vector. We can, and should, normalize the result by dividing by its magnitude. 

7 . 1 . Adjustment of Rotation using Unit Quaternions 

Adjusting the rotation is a little harder. Rotations are conveniently repre¬ 
sented by unit quaternions [Stuelpnagle 1964, Salamin 1979, Taylor 1982, 
Horn 1986, Horn 1987a]. The groundwork for the application of the unit 
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quaternion notation in photogrammetry was laid by Thompson [1959a] and 
Schut [1958-59]. A positive rotation about the axis u; through an angle 9 is 
represented by the unit quaternion 

q = cos(0/2) + sin(0/2) a;, 

where lo is assumed to be a unit vector. Composition of rotations corresponds 
to multiplication of the corresponding unit quaternions. The rotated version 
of a vector r is computed using 

Of o o o * 

r = qrq , 

where q* is the conjugate of the quaternion q, that is, the quaternion obtained 
by changing the sign of the vector part. Here, r is a purely imaginary quater¬ 
nion with vector part r, while r' is a purely imaginary quaternion with vector 
part r'. 

The infinitesimal rotation 8u corresponds to the quaternion 

8u> = 1 + 8u. 

We can adjust the rotation q by premultiplying with 8£j, that is, 

q n+1 = 8u n q n . 

If 8u3 n is not infinitesimal, 8u) n will not be a unit quaternion, and so the result 
of the adjustment will not be a unit quaternion either. This undesirable state 
of affairs can be avoided by using either of the two unit quaternions 

8Cj = \J\ — ||<5u;|| 2 + 8i »>, 

or 

8u = (1 + 8u>)/yj\ + ||<5w|| 2 . 

Alternatively, one can simply normalize the product by dividing by its mag¬ 
nitude. 


7.2. Adjustment of Rotation using Orthonormal Matrices 


The adjustment of rotation is a little trickier if orthonormal matrices are used 
to represent rotations. We can write the relationship 

r' = r + (8u> x r), 


in the form 

r' = r + W r, 

where the skew-symmetric matrix W is defined by 
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in terms of the components of rotation vector 8u = (8u x , 8w y , 8u) z ) T . Conse¬ 
quently we may write r' = Q r, where Q — I + W, or 

1 —8u z 8ui y 

Q = | 8u z 1 —8u x 


-8u, 


Sul, 


One could then attempt to adjust the rotation by multiplication of the ma¬ 
trices Q and R as folows: 

R n+1 = Qn R n 


The problem is that Q is not orthonormal unless 8u> is infinitesimal. In prac¬ 
tice this means that the rotation matrix will depart more and more from 
orthonormality as more and more iterative adjustments are made. It is pos¬ 
sible to re-normalize this matrix by finding the nearest orthonormal matrix, 
but this is complicated, involving the determination of the square-root of a 
symmetric matrix [Horn, Hilden & Negahdaripour 1988]. This is one place 
where the unit quaternion representation has a distinct advantage: it is trivial 
to find the nearest unit quaternion to a quaternion that does not have unit 
magnitude. 

To avoid this problem, we should really start with an orthonormal ma¬ 
trix to represent the incremental rotation. We can use either of the unit 
quaternions 

8& = \Jl — ||<$u;|| 2 + 8(v, 


or 

8& - (1 + 8u)/\jl + ||<5w|| 2 , 

to construct the corresponding orthonormal matrix 

( 00 + 9* “ 9? “ 0* 2 (q x q y - q 0 q z ) 2 (q x q z + q 0 q y ) \ 

2 (q y qx + qoq z ) 9o “ 9x + 9» “ 2(<ly<lz ~ 9o9x) ) , 

2(9x9x - 909s,) 2(q z q y + q 0 q x ) 9o “ 9x ~ <ly + / 

where qo is the scalar part of the quaternion 8<2>, while q x , q y , q z are the com¬ 
ponents of the vector part. Then the adjustment of rotation is accomplished 
using 


R n+1 


Q n R n . 


Note, however, that the resulting matrices will still tend to depart somewhat 
from orthonormality due to numerical inaccuracies. This may be a problem 
if many iterations are required. 


8 . Inherent Ambiguities 

The iterative adjustment described above may arrive at a number of appar¬ 
ently different solutions. Some of these are just different representations of 
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the same solution, while others axe related to the correct solution by a simple 
transformation. First of all, note that —q represents the same rotation as q, 
since 

-q)r(-q ) = qrq . 

That is, antipodal points on the unit sphere in four dimensions represent the 
same rotation. We can prevent any confusion by ensuring that the first non¬ 
zero component of the resulting unit quaternion is positive, or that the largest 
component is positive. 

Next, note that the triple product, [b r{ r r ], changes sign, but not mag¬ 
nitude, when we replace b with — b. Thus the two possible senses of the 
baseline yield the same sum of squares of errors. However, changing the sign 
of b does change the signs of both a and /?. All scene points imaged are in 
front of the camera, so the distances should all be positive. In the presence 
of noise, it is possible that some of the distances turn out to be negative, but 
with reasonable data almost all of them should be positive. This allows us to 
easily pick the correct sense for the baseline. 

Not so obvious is another possibility: Suppose we turn all of the left mea¬ 
surements through 7r radians about the baseline, in addition to the rotation 
already determined. That is, replace rj with 

r',' = 2 (b • r{)b - r}. 

This follows from Rodrigues’ formula for the rotation of a vector r: 

cos 6 r + sin 8 (u> x r) + (1 — cos 9){uj ■ r) oj, 
with 6 — 7r and u) — b. Then the triple product [b r| r r ] turns into 
2 (b • r{)[b b r r ] - [b r\ r r ] = -[b r{ r r ]. 

This, once again, reverses the sign of the error term, but not its magnitude. 
Thus the sum of squares of errors is unaltered. The signs of a and [3 are 
affected, however, although this time not in as simple a way as when the 
sense of the baseline was reversed. 

If [b r\ r r ] = 0, we find that exactly one of a and [3 changes sign. This 
can be shown as follows: The triple product will be zero when the left and 
right rays are coplanar with the baseline. In this case we have 7 = 0, and so 

ar' l = b + /?r r , 

Taking the cross-product with b we obtain 

a (r{ x b) = p (r r x b), 

If we now replace r' f by r" = 2(b • rj)b — rj, we have for the new distances a' 
and /?' along the rays: 

-a'(rj x b) = (3' (r r x b), 

We conclude that the product a 1 (3' has sign opposite to that of the product 
a/3. So if a and f3 are both positive, one of a' or /?' must be negative. 



16 


In the presence of measurement error the triple product will not be ex¬ 
actly equal to zero. If the rays are nearly coplanar with the baseline, however, 
we find that one of a and /? almost always changes sign. With very poor data, 
it is possible that both change sign. (Even with totally random ray directions, 
however, this only happens 27.3% of the time, as determined by Monte Carlo 
methods). In any case, we can reject a solution in which roughly half the 
distances are negative. Moreover, we can find the correct solution directly by 
introducing an additional rotation of ir radians about the baseline. 


9. Remaining Ambiguity 

If we take care of the three apparent two-way ambiguities discussed in the 
previous section, we find that in practice a unique solution is found, provided 
that a sufficiently large number of ray pairs are available. That is, the method 
converges to the unique global minimum from every possibly starting point 
in parameter space. 

Local minima in the sum of squares of errors appear when only a few 
more than the minimum of five ray pairs are available (as is common in prac¬ 
tice). This means that one has to repeat the iteration with different starting 
values for the rotation in order to locate the global minimum. A starting 
value for the baseline can be found in each case using the closed-form method 
described in section 5. To search the parameter space effectively, one needs 
a way of efficiently sampling the space of rotations. The space of rotations 
is isomorphic to the unit sphere in four dimensions, with antipodal points 
identified. The rotation groups of the regular polyhedra provide convenient 
means of uniformly sampling the space of rotations. The group of rotations of 
the tetrahedron has 12 elements, that of the hexahedron and the octahedron 
has 24, and that of the icosahedron and the dodecahedron has 60 (representa¬ 
tions of these groups are given in Appendix A for convenience). One can use 
these as starting values for the rotation. Alternatively, one can just generate 
a number of randomly placed points on the unit sphere in four dimensions as 
starting values for the rotation. 

When there are only five pairs of rays, the situation is different again. In 
this case, we have five non-linear equations in five unknowns and so in general 
expect to find a finite number of exact solutions. That is, it is possible to 
find baselines and rotations that satisfy the coplanarity conditions exactly 
and reduce the sum of squares of errors to zero. If we ignore the three types 
of ambiguities discussed above (incuding the rotation by 7r radians about the 
baseline), then there are generally four distinct sets of baselines and rotations 
that yield an exact solution. Typically only one of these yields positive signs 
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for all of the distances. These are empirical observations; I have not been able 
to prove that there are generally four solutions that satisfy the coplanarity 
conditions. 

10 . Summary of the Algorithm 

Consider first the case where we have an initial guess for the rotation. We 
start by finding the best-fit baseline direction using the closed-form method 
described in section 5. We determine the correct sense of the baseline by 
choosing the one that makes most of the signs of the distances positive. Then 
we proceed as follows: 

• For each pair of corresponding image points, we compute rj t , the left 
ray direction r rotated into the right camera coordinate system, 
using the present guess for the rotation. 

• We then compute the cross-product c, = x r r>t , the double cross- 
product di = rj i x (r rj i x b) and the triple-product ti = [b rj i r r> j]. 

• We accumulate the dyadic products c,c f, c*d J and djd^, as well 
as the vectors t,Cj and The totals of these quantities over all 
image point pairs give us the matrices C, F, D and the vectors c 
and d. 

• We can now solve for the increment in the baseline <5b and the in¬ 
crement in the rotation 8u> using the method derived in section 6. 

• We adjust the baseline and the rotation using the methods discussed 
in section 7, and recompute the sum of the squares of the error terms. 

The new orientation parameters are then used in the next iteration of the 
above sequence of steps. As is the case with most iterative procedures, it is 
hard to decide when to stop. Typically, the total error becomes small after a 
few iterations and no longer decreases at each step, because of limited accuracy 
in the arithmetic operations. One can stop the iteration the first time the error 
increases. Alternatively, one can stop after either a fixed number of iterations 
or after the error becomes less then some predetermined threshold. 

When the decision has been made to stop the iteration, a check of the 
signs of the distances along the rays is in order. If most of them are negative, 
the baseline direction should be reversed. If neither sense of the baseline 
direction yields mostly positive distances, one needs to consider the possibility 
of a rotation through 7r radians about the baseline b. If this also yields mixed 
signs, one is dealing with a local extremum of the error function; something 
that will only happen if the initial guess is in fact not adequate. 

If an initial guess is not available, one proceeds as follows: 
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• For each rotation in the chosen group of rotations, perform the above 
iteration to obtain a candidate baseline and rotation. 

• Choose the solution that yields the smallest total error. 


When there are many pairs of rays, the iterative algorithm will converge to 
the global minimum error solution from any initial guess for the rotation. 
There is no need to sample the space of rotations in this case. Also, instead 
of sampling the space of rotations in a systematic way using a finite group 
of rotations, one can generate points randomly distributed on the surface of 
the unit sphere in four-dimensional space. This provides a simpler means of 
generating initial guesses, although more initial guesses may have to be tried 
than when a systematic procedure is used to sample the space of rotations. 

The method given minimizes the sum of the squares of the triple products 

[b rj r r ]. 


If desired, one can modify it to use some multiple of the triple product as an 
error term by weighting the contribution to the overall sum. This can lead to a 
considerably more complex optimization problem if the weights depend on the 
unknown baseline and the unknown rotation. This happens, for example, if 
we try to minimize the sum of squares of the sines of the angles corresponding 
to vertical disparity: 


sin# — 


[b r} r r ] 

||b x r'|| ||b x r r | ' 


If we assume that the weighting factors vary slowly during the iterative pro¬ 
cess, however, we can to use the current estimates of the baseline and rotation 


in computing the weighting factors, and not take into account the small vari¬ 


ations in the error sum due to changes in the weighting factors. That is, when 
taking derivatives, the weighting factors are considered constant. This is a 
good approximation when the parameters vary slowly, as they will when one 
is close to an extremum. 


11. Critical Surfaces 

In certain rare cases, relative orientation cannot be recovered fully, even when 
there are five or more pairs of rays. Normally, each error term varies linearly 
with distance in parameter space from the location of an extremum, and so the 
sum of squares of errors varies quadratically. There are situations, however, 
where the error terms to not vary linearly with distance, but quadratically or 
higher order, in certain special directions in parameter space. In this case, the 
sum of squares of errors does not vary quadratically with distance from the 
extremum, but as a function of the fourth or higher power of this distance. 
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This makes it very difficult to accurately locate the extremum. In an extreme 
situation, the total error may not vary at all along some curve in parameter 
space. In this case, the total error is unaffected by a small change in the 
rotation, as long as this change is accompanied by a corresponding small 
change in the baseline. There is no localized extremum and consequently 
the solution for relative orientation is not unique. It turns out that this 
problem arises only when the observed scene points lie on certain surfaces 
called Gefahrliche Flachen or critical surfaces [Brandenberger 1947, Hofmann 
1949, Zeller 1952, Schwidefsky 1973]. We show next that only points on 
certain hyperboloids of one sheet and their degenerate forms can lead to this 
kind of problem (see also [Horn 1987b]). 

We could try to find a direction of movement in parameter space (6b, 6u>) 
that leaves the total error unaffected when given a particular surface. Instead, 
we will take the critical direction of motion in the parameter space as given, 
and try to find a surface for which the total error is unchanged. 

Let R be a point on the surface, measured in the right camera coordinate 
system. Then 

3 r r — R and a rj = b + R, 

for some positive a and f). In the absence of measurement errors, 

[b rj r r ] = -\[b (b + R) R] = 0 . 

atp 

We noted earlier that when we change the baseline and the rotation slightly, 
the triple product [b r{ r r ] becomes 

[(b + 6b) (r{ + So; x rj) r r ], 
or, if we ignore higher-order terms, 

[b r\ r r ] + (r', x r r ) • 6b + (r r x b) • (6u> x rj). 

The problem we are focusing on here arises when this error term is unchanged 
for small movement in some direction in the parameter space. That is when 

(r} x r r ) • 6b + (r r x b) • (<W x r{) = 0, 

for some 6 b and 6 w. Introducing the coordinates of the imaged points we 
obtain: 

^ (((b + R) x R) • 6b + (R x b) • (6w x (b + R))) = 0 , 
or 

(R x b) • (6w x R) + (R x b) • (6w x b) + [b R 6b] = 0 . 

If we expand the first of the dot-products of the cross-products, we can write 
this equation in the form 

(R • b)(6w • R) - (b • 6w)(R R) + L R = 0 , 
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where 

L = £ x b, while l = b x 6u> + £b. 

The expression on the left-hand side contains a part that is quadratic in R 
and a part that is linear. The expression is clearly quadratic in X, Y, and 
Z , the components of the vector R = (X,Y, Z) T . Thus a surface leading to 
the kind of problem described above must be a quadric surface [Korn & Korn 
1968]. 

Note that there is no constant term in the equation of the surface, so 
R = 0 satisfies the equation. This means that the surface passes through the 
right projection center. It is easy to verify that R = — b satisfies the equation 
also, which means that the surface passes through the left projection center 
as well. In fact, the whole baseline (and its extensions), R = fcb, lies in the 
surface. This means that we must be dealing with a ruled quadric surface. It 
can consequently not be an ellipsoid or hyperboloid of two sheets, or one of 
their degenerate forms. The surface must be a hyperboloid of one sheet, or 
one of its degenerate forms. Additional information about the properties of 
these surfaces is given in Appendix B, while the degenerate forms are explored 
in Appendix C. 

It should be apparent that this kind of ambiguity is quite rare. This is 
nevertheless an issue of practical importance, however, since the accuracy of 
the solution is reduced if the points lie near some critical surface. A textbook 
case of this occurs in aerial photography of a roughly U-shaped valley taken 
along a flight line parallel to the axis of the valley from a height above the 
valley floor approximately equal to the width of the valley. In this case the 
baseline lies on a circular cylinder that also lies close to the surface on which 
the imaged points lie. This means that it is close to one of the degenerate 
forms of the hyperboloid of one sheet (see Appendix C). 

Note that hyperboloids of one sheet and their degenerate forms are ex¬ 
actly the surfaces that lead to ambiguity in the case of motion vision. The 
coordinate systems and symbols have been chosen here to make the correspon¬ 
dence between the two problems more obvious. The relationship between the 
two situations is nevertheless not quite as transparent as I had thought earlier 
[Horn 1987b]. In the case of the ambiguity of the motion field, for example, we 
are dealing with a two-way ambiguity arising from infinitesimal displacements. 
Here we are dealing with an infinite number of solutions arising from images 
taken with cameras that have large differences in position and orientation. 
Also note that the symbol stands for a small change in a finite rotation 
here, while it refers to a difference in instantaneous rotational velocities in the 
motion vision case. 
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12 . Conclusions 

Methods for recovering the relative orientation of two cameras are of impor¬ 
tance in both binocular stereo and motion vision. A new iterative method for 
finding the relative orientation has been described here. It can be used even 
when there is no initial guess available for the rotation or the baseline. The 
new method does not use Euler angles to represent the orientation. 

When there are many pairs of corresponding image points, the iterative 
method finds the global minimum from any starting point in parameter space. 
Local minima in the sum of squares of errors occur, however, when there 
axe relatively few pairs of corresponding image points available. Method for 
efficiently locating the global minimum in this case were discussed. When only 
five pairs of corresponding image points are given, several exact solutions of 
the coplanarity equations can be found. Typically only one of these yields 
positive distances to the points in the scene, however. This allows one to pick 
the correct solution even when there is no initial guess available. 

All of these methods fail in the rare case that the scene points lie on a 
critical surface. 
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Appendix A—Rotation Groups of Regular Polyhedra 

Each of the rotation groups of the regular polyhedra can be generated from 
two judiciously chosen elements. For convenience, however, an explicit rep¬ 
resentation of all of the elements of each of the groups is given here. The 
number of different component values occuring in the unit quaternions rep¬ 
resenting the rotations can be kept low by careful choice of the alignment of 
the polyhedron with the coordinate axes. The attitudes of the polyhedra here 
were selected to minimize the number of different numerical values that occur 
in the components of the unit quaternions. A different representation of the 
group is obtained if the vector parts of each of the unit quaternions is rotated 
in the same way. This just corresponds to the rotation group of the polyhe¬ 
dron in a different attitude with respect to the underlying coordinate system. 
This observation leads to a convenient way of generating finer systematic sam¬ 
pling patterns of the space of rotations than the ones provided directly by the 
rotation group of a regular polyhedron in a particular alignment with the 
coordinate axes (see also [Brou 1983]). 

The components of the unit quaternions here may take on the values 0 
and 1, as well as the following: 


V5-1 . 1 1 

a = —• 6= 2- c= yi- 


and d = 


y/B+1 


Here are the unit quaternions for the twelve elements of the rotation 
group of the tetrahedron: 


( 1 , 0 , 0 , 0 ) ( 0 , 1 , 0 , 0 ) ( 0 , 0 , 1 , 0 ) ( 0 , 0 , 0 , 1 ) 

(6, 6, 6, 6) (6, b, 6,-6) (6, 6,-6, 6) (6, 6,-6,-6) 

( 6 ,- 6 , 6 , 6 ) ( 6 ,- 6 , 6 ,- 6 ) ( 6 ,- 6 ,- 6 , 6 ) ( 6 ,- 6 ,- 6 ,- 6 ) 

Here are the unit quaternions for the twenty-four elements of the rotation 
group of the octahedron and the hexahedron (cube): 


( 1 , 0 , 0 , 0 ) 
(0, 0, c, c) 
(0, c, c, 0) 
(c, 0, c, 0) 
( 6 , 6 , 6 , 6 ) 
( 6 ,- 6 , 6 , 6 ) 


( 0 , 1 , 0 , 0 ) 
(0, 0, c,-c) 
(0, c,-c, 0) 
(c, 0,-c, 0) 
( 6 , 6 , 6 ,- 6 ) 
( 6 ,- 6 , 6 ,- 6 ) 


( 0 , 0 , 1 , 0 ) 
(0, c, 0, c) 
(c, 0, 0, c) 
(c, c, 0, 0) 
( 6 , 6 ,- 6 , 6 ) 
( 6 ,- 6 ,- 6 , 6 ) 


( 0 , 0 , 0 , 1 ) 
(0, c, 0,-c) 
(c, 0, 0,-c) 
(c,-c, 0, 0) 
( 6 , 6 ,- 6 ,- 6 ) 
( 6 ,- 6 ,- 6 ,- 6 ) 


Here are the unit quaternions for the sixty elements of the rotation group of 
the icosahedon and the dodecahedron: 
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Remember that changing the signs of all the components of a unit quaternion 
does not change the rotation that it represents. 

Appendix B—Some Properties of Critical Surfaces 

In this appendix we develop some more of the properties of the critical sur¬ 
faces. The equation of a critical surface can be written in the form 

(R x b) • (6u> x R) + L • R = 0, 
or 

(R • b)(6w • R) - (b • 6w)(R R) + L R = 0, 

where 

L = £ x b, while £ = b x 8w -f 6b. 

It is helpful to first establish some simple relationships between the quantities 
appearing in the formula above. We start with the observations that £ • b = 0, 
that £ ■ 8u> = 8h ■ 8u, and £ x 8b = — (6b • 6u;)b. 

We can also expand L to yield, 

L = 8u — (b • 6u>)b + 6b x b. 

It follows that L • b = 0, that L • 6b = 8u> • 6b, and 

L x b = —(b x 6u; + 6b) = —£, 

L x 6w = (6b • 6u>)b — (b • 8u>)£. 

We have already established that R = k b is an equation for one of the rulings 
passing through the origin. A hyperboloid of one sheet has two intersecting 
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families of rulings, so there should be a second ruling passing through the 
origin. Consider the vector S defined by 

S = (L x 8u) x L, 
which can be written in the form 

S = (L • L )Slj - (L • 6u) L, 
or 

S = (6u> ■ 8b)£ + (b • 6u)(t • £)b, 

so that S • b = (b • 8u)(£ • £) and S • 8iv = (8u> ■ <5b) 2 + (b • 8u)) 2 (£ ■ £). 

If we substitute R = k S into the formula 


(R x b) • (8u> x R) + L • R, 
we obtain zero, since L • S = 0 and 


S x b = (8u • <5b)L 

is orthogonal to 

S x 8u> = —(L • 8u) L x 8lj. 

We conclude that R = A:S is an equation for the other ruling that passes 
through the right projection center. 

There are two families of parallel planes that cut an ellipsoid in circular 
cross-sections [Hilbert & Cohn-Vossen 1953]. Similarly, there axe two families 
of parallel planes that cut a hyperboloid of one sheet in circular cross-sections. 
One of these families consists of planes perpendicular to the baseline, that is, 
with common normal b. We can see this by substituting R • b = k in the 
equation of the critical surface. We obtain 

k(6u • R) - (b • 8u })(R • R) + L • R = 0, 


or 

(b • <W)(R • R) - (k8u + L) • R = 0. 

This is the equation of a sphere, since the only second-order term in R is a 
multiple of 

R-R = X 2 +Y 2 + Z 2 . 

We can conclude that the intersection of the critical surface and the plane is 
also the intersection of this sphere and the plane, and so must be a circle. 
The same applies to the intersection of the critical surface and the family of 
planes with common normal 8u, since we get 

(b • <5w)(R • R) - (Jfe b + L) • R = 0, 

when we substitute R • 8u = k into the equation of the critical surface. 

The equation of the critical surface is given in the implicit form /(R) = 
0. The equation of a tangent plane to such a surface can be obtained by 
differentiating with respect to R: 

N = (R x 8u}) xb + (Rxb)xfe+L 
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or 

N = (R • b)6u; + (R • 6u>)b - 2(b • 6w)R + L 
The tangent plane at the origin has normal L. This tangent plane contains 
the baseline (since L • b = 0), as well as the other ruling passing through the 
origin (since L • S = 0). Note that the normal to the tangent plane is not 
constant along either of these rulings, as they would be if we were dealing 
with a developable surface. 

In the above we have not considered a large number of degenerate situ¬ 
ations that can occur. The reader is referred to Appendix C for a detailed 
analysis of these. 

Appendix C—Degenerate Critical Surfaces 

There are a number of special alignments of the infinitesimal change in the 
rotation, 6u>, with the baseline, b, and the infinitesimal change in the baseline 
6b that lead to degenerate forms of the hyperboloid of one sheet. 

One of the rulings passing through the origin is given by R = k b, while 
the other is given by R = k S. If these two rulings become parallel, we are 
dealing with a degenerate form that has only one set of rulings, that is a 
conical surface. Now 

S = (6w • 8h)£ + (b • 6w)(/ • £ )b, 

is parallel to b only when (8to • 6b) = 0, since £ is perpendicular to b. In this 
case 

6b • 6u> = 0 and 6b • b = 0, 

so 6b = k(b x 6uj) for some constant k. Consequently £ = (k + l)(b x 8u), 
The vertex of the conical surface must lie on the baseline since the baseline 
is a ruling, and every ruling passes through the vertex. It can be shown that 
the vertex actually lies atR=— (fc + l)b. 

We also know that cross-sections in planes perpendicular to the baseline 
are circles. This tells us that we are dealing with elliptical cones. Right 
circular cones cannot be critical surfaces. It can be shown that the main axis 
of the elliptical cone lies in the direction b + 8io. 

A special case of the special case above occurs when 

||b x 6w|| = 0, 

that is 8uj || b. Here 8u = kb for some constant k and so £ = 6b and 
L = 6b x b. The equation of the surface becomes 

k( R • b) 2 - Jfc(b • b)(R • R) + L • R = 0, 
or 

k ||R x b|| 2 + (6b x b) • R = 0. 
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This is the equation of a circular cylinder with axis parallel to the baseline. 
In essence, the vertex of the cone has receded to infinity along the baseline. 

Another special case arises when the radius of the circular cross-sections 
with planes perpendicular to the baseline becomes infinite. In this case we 
obtain straight lines, and hence rulings, in these planes. The hyperbolic 
paraboloid is the degenerate form that has the property that each of its two 
sets of rulings can be obtained by cutting the surface with a set of parallel 
planes [Hilbert & Cohn-Vossen 1953]. This happens when Suj is perpendicular 
to b, that is, b • Su = 0. The equation of the surface in this case simplifies to 

(R • b)(6u> R) + L R = 0. 

The intersection of this surface with any plane perpendicular to the baseline 
is a straight line. We can show this by substituting 

R • b = k, 

into the equation of the surface. We obtain 

(k 8u + L) • R = 0, 

that is, the equation of another plane. Now the intersection of two planes is 
a straight line. So we may conclude that the intersection of the surface and 
the original plane is a straight line. We can show in the same way that the 
intersection of the surface with any plane perpendicular to Su is a straight 
line by substituting 

R • 8uj = k 

into the equation of the surface. It can be shown that the saddle point of the 
hyperbolic paraboloid surface lies on the baseline. 

A special case of particular interest arises when 8u is perpendicular to 
both b and 6b, that is, 

b • 8u — 0 and 8h • 8u = 0, 

and so 8u = k(8b x b), for some constant k. Then l — (k + 1) 6b and 
L = (k + l)(6b x b). The equation of the surface becomes 

fc(R • b)((6b xb)-R)+(H l)((6b x b) • R) = 0, 

or just 

(k( R • b) + (Jb + 1)) ((8b x b) • R) = 0. 

so either 

(6b x b) • R = 0 or k( R • b) + (k + 1) = 0. 

The first of these is the equation of a plane containing the baseline b and 
the vector 6b. The second is the equation of a plane perpendicular to the 
baseline. So the solution degenerates in this case into a surface consisting of 
two intersecting planes. One of these planes appears only as a line in each of 
the two images, since it passes through both projection centers, and so does 
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not really contribute to the image. It is fortunate that planes can only be 
degenerate surfaces if they are perpendicular to the baseline, since surfaces 
that are almost planar occur frequently in aerial photography. 1 
To summarize then, we have the following degenerate cases: 

• elliptical cones when Suj _L £b, 

• circular cylinders when 8u) || b, 

• hyperbolic parabloids when 6(v _L b, 

• intersecting planes when 8u> ± <5b and 8u _L b. 

For further details, and a proof that not all hyperboloids of one sheet passing 
through the origin lead to critical surfaces, see [Horn 1987b]. 


1 The baseline was nearly perpendicular to the surface in the sequence of pho¬ 
tographs obtained by the Ranger spacecraft as it crashed into the lunar surface. 
This made photogrammetric analysis difficult. 


This blank page ms inserted to preserve pagination. 




CS-TR Scanning Project 
Document Control Form 


Date: 5 I 


Report# Alf>y- 

Each of the following should be identified by a checkmark: 

Originating Department: 

^ Artificial Intellegence Laboratory (Al) 

□ Laboratory for Computer Science (LCS) 

Document Type: 

□ Technical Report (TR) ^J\ Technical Memo (TM) 

□ Other_ 

Document Information Number of pages: 

Not to indude DOD form*, printer intstructions, etc... original pages only. 


Originals are: 
y?( Single-sided or 

□ Double-sided 
Print type: 

n Typsw rit sr I~1 Oflsst Press Q Laser Print 

□ InkJet Printer □ Unknown □ Other 

Check each if included with document: 

DOD Form (ty Q Funding Agent Form 

□ Spine □ Printers Notes 

□ Other_ 

Page Data: 


Intended to be printed as: 
□ Single-sided or 

^.Double-sided 


□ Cover Page 

□ Photo negatives 


Blank Pages** 


pIQI flUNlbtf). 


Photographs/Tonal Material (hyp*,.««*••): 
Other 


Description: Page Number. 

(§ (t)lA,/v-fr J £0 TiTt-iT 


(X'lft Page s It’zto t- 3Q_ 


(39--3H) ,7r^0V- ] boS)f ^t)_ 


_ TRGT) ( 3) 

®)o , CT'h (PasTA' F'Xr'j O'V X f l 


Scanning Agent Signoff: 

Date Received: Date Scanned: £I $ ft 5 


Scanning Agent Signature:. 




Date Returned: _£_/_jLlLi 


Rm WS 4 DSILCS Oocumant ConM Fonn oMonn.«sd 


UNCLASS I FI ED 


SEC'JOITV c _ ASS' c ' C * T iON O r This »»GE Oal* Enter,d) 


REPORT DOCUMENTATION PAGE 

READ INSTRUCTIONS 

BEFORE COMPLETING FORM 

1 REPOST NUMBER 

AI Memo 994 

2. GOVT ACCESSION NO* 


4 TiTlE (end Subtitle) 


s. type of report a period covered 

Relative Orientation 


memorandum 



*. performing org. report number 

7. AUTHOR!*; 

Berthold K.P. Horn 

». CONTRACT OR GRANT NUMBER! •; 

DACA76-85-C-0100 

N00014-85-K-0124 

» performing ORGANIZATION name and address 

Artificial Intelligence Laboratory 

545 Technology Square 

Cambridge, MA 02139 

10. PROGRAM ELEMENT. PROJECT, TASK 
AREA A WORK UNIT NUMBERS 

M. CONTROLLING OFFICE NAME ANO ADORESS 

Advanced Research Projects Agency 

1400 Wilson Blvd. 

Arlington, VA 22209 

12. REPORT DATE 

September 1987 

IS. NUMBER OF PAGES 

31 

14 MONITORING AGENCY NAME A AOORESSrff dllltt, nl frost Controlling Otile*) 

Office of Naval Research 

Information Systems 

IS. SECURITY CLASS, tot 'hi, raperf; 

UNCLASSIFIED 

Arlington, VA 22217 


t»a. OECL ASSIFICATION/DOWNGRADING 
SCHEOULE 

16. DISTRIBUTION STATEMENT (of thie Report) 



Distribution is unlimited. 



17* DISTRIBUTION STATEMENT (o( the eb effect mMeted in Bloek 20, if dlilerent item Report) 

is. supplementary notes 



None 



ft* KEY WOROS (Continue on roooroe eldo II neoeeeory ipentttj by block member) 

relative orientation photogrammetry 

binocular stereo motion vision 

coplanarity condition representation of rotation 


20. ABSTRACT (Continue on raaaraa old# It rtacaaaarr and Idartflfji 01,tk imtafj 


Before corresponding points in images taken with two cameras can be used 
to recover distances to objects in a scene, one has to determine the position 
and orientation of one camera relative to the other. This is the classic 
photogrammetric problem of "relative orientation," central to the interpretaticn 
of binocular stereo information. Iterative methods for determining relative 
orientation were developed long ago; without them we would not have most of th« 
topographic maps we do today. Relative orientation is also of importance 

DD ,j°n M 73 1473 COITION OF I NOV SS IS OBSOLETE UNCLASSIFIED 

S/N 010 2-014* 660 1 I 

SECURITY CLASSIFICATION OF THIS rase rattan Data Mnlotoe 















Block 20 cont. 


in the recovery of motion and shape from an image sequence when successive 
frames are widely separated in time. Workers in motion vision are redis¬ 
covering some of the methods of photogrammetry. 

Described here is a particularly simple iterative scheme for recovering 
relative orientation that, unlike existing methods, does not require a good 
initial guess for the baseline and the rotation. The data required is a set 
of pairs of corresponding rays from the two projection centers to points 
in the scene. It is well known that at least five pairs of rays are needed. 
Less appears to be known about the existence of multiple solutions and their 
interpretation. These issues are discussed here in detail. The unambiguous 
determination of all of the parameters of relative orientation is not 
possible when the observed points lie on a critical surface. These surfaces 
and their degenerate forms are analysed here as well. 



Scanning Agent Identification Target 


Scanning of this document was supported in part by 

the Corporation for National Research Initiatives, 
using funds from the Advanced Research Projects 
Agency of the United states Government under 
Grant: MDA972-92-J1029. 


The scanning agent for this project was the 
Document Services department of the M.I.T 
Libraries. Technical support for this project was 
also provided by the M.I.T. Laboratory for 
Computer Sciences. 


Scanned 

Da,e: i/s/n^s 

M.I.T. Libraries 
Document Services 


darptrgt.wpw Rev. 9/94 


